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The purpose of this study is to examine the impact of training program characteristics on training 
effectiveness among organizations receiving training services from external training providers. Two 
surveys were sent to HRD managers and senior managers per company. The results showed that the 
operational margin of the programs where private training providers were involved increased more and 
that senior managers perceived the entirely developed training program was more effective than the 
generic, standard programs. 
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Rapid changes in the business environment and competitive market conditions have required business organizations 
to sustain their competitiveness through employee development (Jacobs, 2003). Business organizations have 
emphasized the significance of training and made great efforts to improve training quality in order to sustain 
competitiveness and improve performance (Jacobs & Washington, 2003). However, due to limited resources, 
business organizations have increasingly utilized external sources for identifying training needs, and for developing 
and implementing training programs (Knoke & Janowiec-Kurle, 1999). 

Business organizations acquire training related services from external training providers such as training 
agencies, consultants, and educational institutions (Knoke, 1997; Sole 1999). Among outside training providers, 
community colleges have actively provided a range of training related services. The characteristic of partnership 
training between external training providers and client organizations has become one of the major factors that can 
influence the impact of training (Hardingham, 1996). 

As outsourced training has become more common, the nature of the relationships among client organizations 
and external training providers, especially in partnership training, varies (Hawley et ah, 2005). Some outsourced 
training programs with community colleges have formal contracts while others may not. Some participants in the 
partnership training complete the training program with or without academic credentials such as certificates, licenses, 
or degrees. External training providers’ level of involvement and their history of the relationships might vary. In 
addition, different types of training providers add to the diversity of the relationship among client organizations and 
external training providers. 

It is widely believed that outsourced training is beneficial to business organizations (Gold et ah, 1998; Hall & 
Scott, 2001; Hardingham, 1996; Mavin & Bryans, 2000; Roever, 2000; Ryan & Heim, 1997). Business 
organizations often have outsourced training programs simply because of the need to develop a full range of training 
programs to meet organization’s needs. Previous research on outsourced training has found that training programs 
serve various needs including organizational development and employee development. (Aslanian, 1988; Johnstone, 
1994; Roever, 2000; Ryan & Heim, 1997) 

As outsourced training has become more widespread, the training needs assessments vary. Business 
organizations have different training needs and resources and have their own training design process. At the same 
time, external training providers also have different resources and experience in training design process and 
implementation. Thus, some training design processes in partnership training programs heavily involve needs 
assessment while others may not. Therefore, partnership training is widely implemented with various quality of 
training needs assessment. 

It is also believed that the outcomes of training for organizations are dependent not only on the quality of the 
training needs assessment and the nature of the training program such as the type of training and extent of 
customization, but also on the nature of the relationships with external training providers. Although the outsourced 
training provided by external training providers has unique characteristics in the nature of the relationship among 
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client organizations and external training providers, training needs assessment, and the nature of the training 
program, there is little information about whether those unique relationships, training needs assessment, and the 
nature of the training program relate to any outcomes of training. In other words, literature provides little 
information whether the training effectiveness differ if the nature of the provider-client relationships, training needs 
assessment, and the nature of the training programs differ in outsourced training. 

Therefore, the purpose of this study is to examine the impact of training program characteristics on training 
effectiveness among organizations receiving training services from external training providers. It is to evaluate 
training effectiveness as a function of the nature of the relationships among client organizations and external training 
providers, the training needs assessment, and the nature of the training programs. 

Theoretical Framework 

The dependent variable in this study is training program effectiveness, which refers to whether training achieves its 
intended purpose or goal. Training effectiveness can be measured either by trainees or by supervisors at an 
individual level, thus the aggregated data are often reported as training effectiveness (Ahlstrand et al., 2003; 
Brinkerhoff & Gill, 1994; Kirkpatirck, 1996). 

The outcomes of outsourced training for client organizations are dependent not only on the quality of training 
needs assessment and the nature of the training program, but on the nature of how business organizations have 
developed relationships with training providers. Partnerships between client organizations and external training 
providers contribute to the quality of the training through three main factors: the nature of the provider-client 
relationship, provider's training needs assessment, and the nature of the training program. Based on the literature 
review, the following hypotheses are developed to conduct this study. 

Formal partnership contracts are hypothesized to create benefits for firms (Hawley, 2003; Kenis & Knoke, 
2002). By having formal contracts, several training conditions including training programs, training content, 
expected outcomes, and trainers’ or trainees’ pre-requisites can be written more explicitly. In addition, a formal 
relationship represents management’s level of involvement and commitment. Because many studies have proved 
that management’s support is one of the strongest factors for success of training programs, formal relationships may 
be a critical factor to have successful implementation (O’Rear, 2002). It can be hypothesized that training programs 
with external training providers who have a formal contract and exchange a company’s strategy are more relevant to 
training program effectiveness than with external providers without a formal relationship. In addition, if the external 
training providers are involved in more stages of training process, have more previous contracts, and initiate a 
follow-up contact, it is hypothesized that training effectiveness is higher. 

While training needs assessment is significant and most training providers implement some form of training 
needs assessment, needs assessment does not automatically guarantee training effectiveness (Brown, 2002). The 
level of and quality of actual needs assessment might increase training effectiveness and its impact on organizational 
performance (Goldstein, 1993; Witkin & Altschuld, 1995). Thus, we can hypothesize that more levels of focus 
(among individual, work process, and function) of provider's training needs assessment increases perceptions of 
training effectiveness. In addition, it can be hypothesized that if a needs assessment identifies performance levels 
before and after training, the needs assessment results in greater perceptions of training program effectiveness and 
higher than when the needs assessment either identifies performance levels before training or after training. 

The nature of the training program is also hypothesized to have a relationship with perceptions of training 
effectiveness (Clarke, 1984). It is believed that if training programs utilize the company’s data, format, materials, 
and/or equipments for instructional materials, trainees can learn more and can easily apply this learning to their 
work place (Brown, 1999). Thus, we can hypothesize that the higher level of customization provided among 
partnership training programs, the more relevant they are to training program effectiveness (Moore, et al., 2003). 
The level of customization can be defined three ways: (1) generic/standard training program with little 
customization, (2) moderate level of revision, and (3) newly developed training programs. The areas to be 
customized are components of a lesson plan. They are training purpose, trainee’s pre-requisites, content, delivery 
methods, and evaluation. Thus, it can be hypothesized that more customized training programs increase perceptions 
of training program effectiveness. In addition, the job-specific training programs are hypothesized to have higher 
perceptions of training effectiveness than the job-related training programs. 

Research Questions 

1. Does training program effectiveness differ based on types of training providers? Does the degree of 
training program effectiveness differ based on the nature of the relationship among client organizations and 
external training providers? 
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2. Does the degree of training program effectiveness differ based on the quality of provider’s training needs 
assessment? 

3. How do training programs vary in terms of the extent of customization, type of training, relationship to job, 
proportion of participants verses total employees, and expected outcome? How do these differences in the 
nature of the training program impact on training program effectiveness? 

Methodology and Research Design 

Research Sample 

The research sample is all the companies that implemented and completed training programs funded by the 
Ohio Investment in Training Program (OITP) from January 2002 to June 2004. OITP was established to help 
businesses in Ohio enhance their organizational competitiveness and to increase their organization’s performance 
through trainings. OITP companies who received training dollars were used for this study. 

This study used primary data and secondary data for analysis. The primary data was gained through survey to 
the OITP fund recipient companies. The sampling frame was obtained from the current OITP database, which was 
updated the 1 st of October, 2004. Since the OITP first provided their training fund for companies under the name of 
the OITP in 1999, 432 companies have participated and are currently listed in the OITP database. The secondary 
data was gained through an analysis of OITP database. The Economic Development Division, Ohio Department of 
Development maintains and updates the OITP database which contains generic information about OITP participant 
companies and their training programs. The analysis of the OITP database provided a sampling frame to draw a 
survey list and basic participant’s demographic information as well as basic participant training information. It 
provided information such as the company’s training coordinator’s contact information either or both of phone 
number and email address, company name, company's geographic location, company size, industry sector with SIC 
code number, company ownership, the type of training, and training start and end dates. 

Data Collection 

The data was collected through web-based survey to HRD manager or training coordinators and through 
individualized web-based survey to matching senior managers. 

The total number in the sample frame is 125. Fifty seven out of 125 companies replied to the first survey, the 
manager survey. Forty five companies out of fifty seven replied to the second survey, the senior manager survey. As 
a result, 45 companies out of 125 completed both surveys, resulting in a response rate of 36 percent. 

No significant difference was found among the three groups — respondents, non-respondents, and not-in-sample 
frame companies — in terms of company location, size, and industry type. In addition, no significant differences were 
found when comparing the three groups in terms of the number of training participants and the average of base wage 
of training participants. 

Data Analysis 

After the data was collected, it was automatically coded and could be easily transferred to the spreadsheet 
database. The data from the secondary data sources was merged in one database and analyzed by using the 
Statistical Package for the Social Science (SPSS). 

First, descriptive statistics was generated to summarize the proportion of different types of training providers in 
partnership training. Frequencies and percentages were calculated. One-way ANOVA was applied to examine group 
differences in the degree of perceived training effectiveness among different types of training providers. The F value 
was calculated to see the difference. Standard multiple regression analysis was performed to examine the 
relationship between perceived training effectiveness and the nature of the relationship. The correlation statistics 
were used to determine the relationship between perceived training effectiveness and the quality of training needs 
assessment. Descriptive statistics were generated to present how the nature of the program varies regarding the five 
sub variables. Frequencies, percentages, mean, median, range, and standard deviations were calculated. And then, 
standard multiple regression analysis was employed to examine the relationship between perceived training 
effectiveness and the nature of the program. The model regressed the degree of the perceived training effectiveness 
on a vector of five sub variables. 

Results and Findings 

Respondents 

From this part, the researcher analyzed 45 completed cases from both manager and senior manager surveys. 
Most participant companies (73.3%) are classified as “Manufacturing” according to SIC code. The remaining 
companies represent the service and trade industry (Table 4.3). Seven participant companies (15.6%) are either 
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publicly owned or subsidies of publicly owned companies, and the majority (84.4%) are either privately owned or 
subsidies of privately owned companies. Participant companies are located throughout Ohio. Five companies 
(11.1%) are located in Cuyahoga county, and 4 companies (8.9%) are located in Summit county. The majority of 
companies are located throughout the state in 21 counties ranging from one, two, or three from each county. Figure 
1 shows survey participant companies distribution in counties of Ohio. 

Table 1 describes training program characteristics. Forty four percent of the training programs (20 programs) 
include technical training and managerial training. Awareness training makes up 20 percent of the programs. The 
length of training programs ranges from short to long with 33 percent being longer than 8 weeks, and 27 percent 
being shorter then 2 weeks. The length of training programs shape is bimodal. The mean training duration is 13.48 
weeks (SD=18.40). 

The number of participants in training ranges from 3 to 1,210 (M=99 people, SD=210). Less than 30 people 
participate in half of the training programs (53.3%). All levels of employees (frontline, supervisor, manager, and 
executive) participate in 5 training programs (11.1%). Only 12 training programs (26.7%) have one level of 
participant, and the remaining programs have more than one level of employee participating. 



Table 1 : Demographic Information of the Training Programs (Manager survey, n=45) 





n 


% 


Types of Training 


Technical 


20 


44.4 


Managerial 


10 


22.2 


Awareness 


9 


20.0 


Missing 


6 


13.3 


Duration 


0-2 weeks 


12 


26.7 


3-4 weeks 


3 


6.7 


5-8 weeks 


5 


11.1 


More than 8 weeks 


15 


33.3 


Missing 


10 


22.2 


Number of Participants 


1-9 employees 


7 


15.6 


10-19 employees 


8 


17.8 


20 - 29 employees 


4 


8.9 


30- 39 employees 


6 


13.3 


40 - 49 employees 


4 


8.9 


50 - 99 employees 


8 


17.8 


100 - 999 employees 


7 


15.6 


Over 1,000 employees 


1 


2.2 


Trainees’ Employment Level in Company* 


Frontline employee 


40 


88.9 


Supervisor 


30 


66.7 


Manager 


24 


53.3 


Executive 


10 


22.2 


Number of Employment Level Participated in Training 


1 Level 


12 


26.7 


2 Levels 


12 


26.7 


3 Levels 


16 


35.6 


4 Levels 


5 


11.1 



* Companies can answer more than one category as their trainees’ level in companies. 



Training program effectiveness, types of training providers, and the relationship among client organizations and 
external training providers 

Training programs were developed and delivered by various entities. The majority of training programs were 
developed (66.7%), and delivered by private companies (55.6%), however, some were developed and delivered by 
cooperation with in-house training staff within client companies. Compared to private companies, educational 
institution involvement in training development and delivery is relatively small (11.1%). Private training providers 
include private training vendors and equipment manufacturers. Educational institutions include two-year colleges, 
four-year universities, and vocationaktraining center. 
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One-way analysis of variance was applied to examine group differences among the degree of training 
effectiveness by different types of training providers. Table 2 presents group differences in the degree of perception 
on the specific training program effectiveness. Table 2 shows the value of F (2, 41) is .037 and is not significant 
(p>. 10), which implies that there is no significant difference among training developer. Another value of F (2, 41) 
is .901 and is not significant (p>.10), which implies that there is no significant difference based on training deliverer. 
Table 2. One-way ANOVA on Type of Training Provider-Perception on Effectiveness of this Specific Program 



Source 


Sum of Squares 


Df 


Mean Square 


F 


Sig. 


Training developer 


.031 


2 


.016 


.037 


.964 


Error 


17.230 


41 


.420 






Total 


17.261 


43 








Training deliverer 


.088 


2 


.044 


.901 


.901 


Error 


17.173 


41 


.419 






Total 


17.261 


43 









(Manager survey and Senior manager survey, n=43) 



Table 3 presents the results of independent samples t-test regarding private training provider involvement. The t 
value in terms of perception on this specific training program effectiveness is .052 and is not significant (p>.10). 
Flowever, the t value in terms of increase in operational margin is 2.888 and is significant (p>.05). This implies that 
there is a statistical difference in the increase in operational margin between the two groups. 

Thus, the results show that client organization’s perceived training effectiveness does not differ based on the 
type of training providers. Flowever, the results show that there is a statistical difference in the increase in 
operational margin between the training programs where private training providers were involved versus the 
programs that did not involved private training providers. 

Table 3. Independent Samples t-test on Private Training Providers and Training Program Effectiveness (Manager 



survey and Senior manager survey) 


Training t 

Effectiveness 


df 


Sig. 


Mean Difference 


SE 


Perception of this 
training program .052 

(n=43) 


42 


.959 


.011 


.204 


* p<.05 



Standard multiple regression analysis was performed on both manager and senior manager surveys to examine 
the relationship between training program effectiveness and the nature of the relationship with external training 
providers. The model regressed the degree of the training program effectiveness on a vector of five independent 
variables: external training provider’s level of involvement in training process, contract history with this particular 
training provider, whether the providers initiated a follow-up contact, the external training provider’s knowledge and 
experience in client organization’s business, and degree of formality of the contract. 

Table 4 presents the results of standard multiple regression analysis. For perception of this specific training 
program effectiveness, none of the standardized coefficients (beta) of independent variables shows a significant 
relationship with this perception at an alpha level of .05 (Table 5). The squared multiple coefficient of correlation 
(Adjusted R square) is 0. 



Table 4: Standard Multiple Regression of Perception of the Specific Training Program Effectiveness on Selected 
Variables in the Relationship with External Training Providers (n = 28) (Manager survey and Senior manager 
survey) 



Variables 


R 2 


R 2 change 


b 


Beta 


Provider’s Level of Involvement 


.017 


.017 


-.113 


-.187 


Contract Flistory with Providers 


.036 


.019 


.080 


.315 


Provider’s Knowledge & Experience 


.107 


.071 


-.664 


-.676 


Provider’s Follow-up 


.108 


.000 


-.035 


-.020 


Contract 


.221 


.113 






Contract 1 - neither written nor verbal 






-1.839 


-.513 


Contract II- first written, later verbal 






.157 


.107 


Contract III- written contract in every time 






-.100 


-.062 


(Constant) 






7.253 





Standard error = .678 
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Training program effectiveness and training needs assessment 

The result of the t-test shows there is no significant difference between the two groups. The t value in terms of 
perception on this specific training program effectiveness is -.769 and is not significant (p>. 10). The correlation 
coefficient between the perception of the specific training program effectiveness and the quality of training needs 
assessment is .175, and is not statistically significant (p<.10). Thus, the results show that training program 
effectiveness does not differ based on the quality of training needs assessment. 

Training program effectiveness and the nature of the training program 

First, descriptive statistics are presented in table 1 and table 5 to show how the training program varies 
regarding the five sub variables. As table 1 shows, the type of training varies from technical training (44.4%), to 
managerial training (22.2%) and awareness training (20%). The proportion of training participants among total 
employees also varies. Table 7 shows that 11 training programs (24.3%) have less than 10 percent of participants 
versus total employees, and that all employees participated in 13 training programs (28.9%). For the remaining 21 
training programs participation varies between 10 and 99 percent. The average participants’ proportion among total 
employees is 50 percent (SD=40.0 %). 

Table 5 shows that most training programs are job-specific (n=32, 71.1%) rather than job-related (n=13, 28.9%). 
Almost half (44.4%) of the training program’s intended goal is related to “to increase knowledge and skills”, and the 
other half (42.2%) reported ones to be related to “increasing organizational performance” as the intended training 
program goal. Other specified training goals are “ISO certificate” and “to develop and sustain a positive culture.” 

In terms of customization, only one training program has never customized any part of the training program, 
and the other 44 training programs customized at least more than one item of the training program with client 
companies materials or to meet participant’s needs. The average number of customized items per training program is 
5.67 (SD=2.67) among 10 items. Among customized items, instructional materials are most frequently customized 
(n=39, 86.7%), and training objectives or trainees’ prerequisites are customized next frequently (n=36, 80%). Table 
4.22 shows that 35 training programs (77.8%) were developed with some level of customization. 



Table 5: Frequencies and Percentages on Intended Goal of Training, Level of Relationship to Job, and Level of 
Customization (Manager survey, n=45) 





N 


% 


Intended Goal of Training 


Increase Knowledge & Skills 


20 


44.4 


Change Behavior 


4 


8.9 


Increase Organizational Performance 


19 


42.2 


Other 


2 


4.4 


Relationship to Job 


Job-specific 


32 


71.1 


Job-related 


13 


28.9 


Extent of Customization 


No customization 


1 


2.2 


Customized 


9 


20.0 


Entirely developed 


35 


77.8 



Table 6 present the results of standard multiple regression analysis. For perception of this specific training 
program effectiveness, six standardized coefficients of independent variables shows statistically significant 
relationship with this perception at the alpha level of .05 (Table 4.25). First, the beta of portion of trainees among 
employees is -.344 (p<.05), which means that if the number of trainees among employees increases, perception of 
this specific training program effectiveness can be predicted to decrease. Second, the beta of intended goal to 
increase learning is -1.230 (p<.05), which means that if the intended goal is to increase learning, perception on this 
specific training program effectiveness can be predicted to decrease. The beta of intended goal to change behavior is 
-1.178 (p<.01), which also means that if the intended goal is to change behavior, perception of this specific training 
program effectiveness can be predicted to decrease. The beta of intended goal to increase organizational 
performance is -1.244 and is significant (p<.05). The beta of entirely development of the program is .469 (p<.01), 
which implies that if the training program is entirely developed, perception of this specific training program 
effectiveness is predicted to increase. Flowever, the beta of extent to customization is -.393 (p<.05), which means 
that if the number of customized training parts increases, perception of this specific training program effectiveness is 
predicted to decrease. The squared multiple coefficient of correlation is .376, which means that the group of 
independent variables explains 37.6 percent of the total variance of the perception of this specific training 
program’s effectiveness. 
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Table 6: Standard Midtiple Regression of Perception of the Specific Training Program Effectiveness on Selected 
Variables in Teaming Program Characteristics (n = 37) (Manager survey and Senior manager survey) 



Variables 


R 2 


R 2 change 


b 


Beta 


Proportion of trainees among 
employees 


.124 


.124 


-.592 


-.344* 


Job specific 


.162 


.038 


.209 


.143 


Intended goal 


.332 


.170 






Intended goal I 






-1.665 


-1.230* 


Intended goal II 






-2.541 


-1.178** 


Intended goal III 






-1.649 


-1.244* 


Entirely developed 


.408 


.076 


.704 


.469** 


Level of customization 


.525 


.117 


-.095 


-.393* 


Type of training 


.527 


.003 






Type of training I 






-.084 


-.063 


Type of training II 






-.117 


-.075 


(Constant) 






6.782 





Standard error = .530 



Conclusions and Recommendations 

The result of this study identified a way to evaluate partnership training relationship, like training provider’s level of 
involvement in training process and training provider’s knowledge in client organization’s business. Based on 
previous studies, it is not surprising that training provider’s level of involvement and knowledge was positively 
correlated with training effectiveness. However, in contrast to previous studies (Kenis and Knoke, 2002), formal 
contracting was negatively correlated with training effectiveness. This result may be explained by the small number 
of case engaging with external training providers in the sample. 

in addition, the result of this study showed that training needs assessment did not relate to training effectiveness 
of the companies that received training funds from the OITP, contrary to the results of previous research on needs 
assessment. Since the study represents one setting, it’s difficult to generalize the results to other contexts. 

Although the number of items customized was identified not to relate to training effectiveness, the result of the 
study showed that the training programs that were entirely developed showed a large increase in operational margin 
at client organizations. Because most training programs customized at least some instructional materials such as 
company data, format, materials, and equipment, the extent of customization may not be related to training 
effectiveness. 

In terms of training duration, the distribution was bi-modal with the average of 13.48 weeks. Since the results of 
study do not provide any information regarding intensity of training program — how many hours per week — , the 
amount of training duration has limits to be tested for training effectiveness. Another limit is that the absolute 
number of case is too small in certain variables to apply statistical analysis. This study also cannot eliminate other 
training program’s impact besides the OITP funded one. 

This study also identified needs to further study in partnership training. This study identified business 
organizations frequently utilize private training providers. It needs to study further about why private training 
providers are preferred. It is also important to study these reasons are relate to organizational level of performance. 
It is essential to identify more variables that impact external training providers and, ultimately, impact training 
effectiveness to increase organizational performance. This study identified external training provider’s level of 
involvement in training process and the provider’s knowledge about client organization’s business. However, the 
study failed to clearly identify how the contractual relationship between client organizations and external training 
providers impact on training effectiveness. If the contractual relationship might impact on training effectiveness, it is 
very critical to identify other contractual components. It might be more appropriate if qualitative analysis is emerged 
to investigate the relationship. In addition, this study attempted to examine the quantity of interaction with external 
training providers. A case study is recommended to study the quality of interaction in addition to the quantity of 
interaction, to more clearly understand the impact of the relationship between external training providers and 
training effectiveness. 

In terms of training needs assessment, this study showed that implementing training needs assessment does not 
relate to training effectiveness. Thus, rather to identify whether training needs assessment is implemented or not, it 
might be more meaningful to study the quality of training needs assessment. Qualitative research methods can be 
used to identify determining factors of quality. 
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